At its annual GTC conference for AI developers, Nvidia today announced its next-gen Hopper GPU architecture and the Hopper H100 GPU, as well as a new data center chip that combines the GPU with a high-performance CPU, which Nvidia calls the "Grace CPU Superchip" (not to be confused with the Grace Hopper Superchip). With Hopper, Nvidia is launching a number of new and updated technologies, but for AI developers, the most important one may just be the architecture's focus on transformer models, which have become the machine learning technique de rigueur for many use cases and which powers models like GPT-3 and asBERT. The new Transformer Engine in the H100 chip promises to speed up model training by up to six times and because this new architecture also features Nvidia's new NVLink Switch system for connecting multiple nodes, large server clusters powered by these chips will be able to scale up to support massive networks with less overhead. "The largest AI models can require months to train on today's computing platforms," Nvidia's Dave Salvator writes in today's announcement. AI, high performance computing and data analytics are growing in complexity with some models, like large language ones, reaching trillions of parameters.

Did you miss a session at the Data Summit? Nvidia offered details on its Grace central processing unit (CPU) "Superchip" during CEO Jensen Huang's keynote speech at its virtual Nvidia GTC 2022 event. Huang said the chip would double the performance and energy efficiency of Nvidia's chips. It is on schedule to ship next year, he said, and it can be a "superchip," or essentially two chips connected together. The chip is Nvidia's own variant of the Arm Neoverse architecture, and it is a discrete datacenter CPU designed for AI infrastructure and high-performance computing, providing the highest performance and twice the memory bandwidth and energy-efficiency compared to today's leading server chips, Huang said.